1. Field of the Invention
The present invention relates to an image processing device for converting image signals having various formats to image signals in a desired format or for synthesizing the image signals.
2. Related Background Art
The development of multimedia in recent years provides occasions in which images in various image signal formats are displayed on a display screen. Particularly, while a display of a TV set had once been quite different from that of a personal computer (PC), a TV capable of displaying PC images and a PC display to which TV signals can be entered have appeared due to an advancement of their unification. In addition, due to an appearance of video sources in new digital formats such as a digital television or MPEG or due to an advancement of three-dimensional graphics, motion picture is more frequently displayed also on PC displays.
FIG. 6 shows a block diagram of the conventional display. In this drawing, there are shown an analog image signal input terminal 1-1, a horizontal synchronizing signal (IHD) input terminal for input signals 1-2, a vertical synchronizing signal (IVD) input terminal for input signals 1-3, an AD converter 2 for converting analog image signals inputted to the input terminal 1-1 to n-bit digital signals, an input system image processing unit 3, a memory control unit 4, a memory unit for storing image data 5, an output system image processing unit 6, an image display unit 7, data buses 20-1, 20-2, 20-3, and 20-4 for transmitting n-bit digital signals to respective units, a control bus 21 comprising memory control lines and address lines, and a memory data bus 22.
Reference numeral 8 designates a phase locked loop (PLL) circuit and reference character ICK designates an input system clock synchronized with an input IHD. Further, there are shown an oscillator circuit 12 which generates output system clocks OCK, an H-counter and V-counter circuit 11 which generates output system horizontal synchronizing signals OHD and vertical synchronizing signals OVD based on the output system clock OCK, a microcomputer (μCOM) unit 9, and m control buses for controlling respective units 19.
Digital image signals are subjected to image quality adjustment, image reducing conversion, or other processing in the input system image processing unit 3 before being stored in the memory unit 5 and then transferred to the memory control unit 4. The memory control unit 4 stores image data into the memory unit 5 at a timing corresponding to an input synchronizing signal (IHD, IVD) and an input system clock ICK and transfers the image data from the memory unit 5 to the readout output system image processing unit 6 at a timing corresponding to a horizontal synchronizing signal OHD and a vertical synchronizing signal OVD. The image processing unit 6 adjusts image qualities and converts images with enlargement. Accordingly, input images having various input system formats are converted to image signals having a format appropriate for the image display unit 7 via the memory.
Furthermore recently in a wide-screen display device such as a wide-screen television, a plasma display, a rear projection-type TV, or a front projection-type projector, there has been an increase of situations in which various video sources such as films, televisions, home videos, presentations, TV conferences, and displays of various materials are used in offices or houses. In addition among these types of displays, there is a display having a multi-screen display function of displaying a plurality of images from different input sources on a single screen with the screen divided.
FIG. 13 shows as an example of a display conventionally used in this situation a block diagram of an image processing unit of an image display device having one system for an input of digital computer image signals and the other system for two-system PC inputs which are analog computer image signal inputs, in which outputs of a frame memory are controlled for synchronization so as to perform a multi-screen display having two screens in a single-system image display unit.
In FIG. 13, there is shown an input terminal 1-1a for q-bit digital computer image signals (IDATA1) of a first system (PC1). While intrinsically a description should be made for three systems of red, blue, and green (RGB), only one system is used here for a simple description of a configuration (the same shall apply hereinafter). There are also shown an input horizontal synchronizing signal (IHD1) input terminal 1-1b, an input vertical synchronizing signal (IVD1) input terminal 1-1c, an image signal clock (ICK1) input terminal 1-1d, and a DDC (DDC1) input-output terminal 1-1e, data buses 20-1a-1 and 20-1a-2 for transmitting q-bit digital image signals to respective units. In addition, there are shown IHD1, IVD1, ICK1, and DDC1 signal lines 20-1b, 20-1c, 20-1d, and 20-1e, respectively.
DDC is a standard for communication means for computers to recognize or control displays recommended by a standardizing organization, a Video Electronic Standard Association (VESA).
There are also shown an input terminal 1-2a for analog computer image signals (IDATA2) of a second system (PC2), an input horizontal synchronizing signal (IHD2) input terminal 1-2b, an input vertical synchronizing signal (IVD2) input terminal 1-2c, and a DDC (DDC2) input-output terminal 1-2e. 
An AD converter 2 converts an analog image signal (IDATA2) to an n-bit digital signal. A PLL circuit 8 generates input system clocks (ICK2) in a PC2 side synchronized with horizontal synchronizing signals (IHD2) inputted from the terminal 1-2b. 
Reference numeral 20-2a-0 designates an analog signal line and reference numerals 20-2a-1 and 20-2a-2 designate n-bit digital signal lines. Reference numerals 20-2b, 20-2c, 20-2d, and 20-2e designate signal lines for IHD2, IVD2, ICK2, and DDC2, respectively.
There are also shown a PC1 input system image processing unit 3-1, a memory control unit 4 for a control of storing image signals inputted from the two-system input image processing units in a memory once and synthesizing images and outputting them to an output system image processing unit so as to output them on a multi-screen, frame memories (memory A, memory B) 5-1 and 5-2 corresponding to input systems PC1 and PC2, respectively, control buses 21-1 and 21-2 for the memory A and the memory B, respectively, and data buses 22-1 and 22-2 for the memory A and memory B, respectively.
A microcomputer unit 9 controls a system and microcomputer buses (MB) 19-1 and 19-2 comprise control lines from a microcomputer to respective units and data lines.
An oscillator circuit 12 generates output system clocks (OCK).
A H-counter and V-counter unit 11 counts output system clocks (OCK) and generates horizontal synchronizing signals (OHD) and vertical synchronizing signals (OVD) of the output system.
Additionally there are shown an output system image processing unit 6 and an image display unit 7 such as a plasma display and a CRT.
Reference characters 1-f, 1-g, 1-h, 1-i, and 1-j designate an input terminal of an image display unit for image display digital data (ODATA), an input terminal of an image display unit for output horizontal synchronizing signals (OHD), an input terminal of an image display unit for output vertical synchronizing signals (OVD), an input terminal of an image display unit for output image signal clock (OCK), and an input terminal of an image display unit for microcomputer buses (MB), respectively.
Furthermore, 20-f-1, 20-f-2, and 20-f3 are signal lines for k-bit ODATA and 20-g-1 and 20-g-2 are signal lines for OHD. 20-h-1 and 20-h-2 are signals lines of OVD and 20-i-1 and 20-i-2 are signal lines for OCK.
Digital image signals inputted from the image input terminal 1-1a are stored into the memory unit A designated by 5-1 after being subjected to an image quality adjustment, reducing conversion, and other processing in the input system image processing unit 1 designated by 3-1 and then transferred to the memory control unit 4.
Analog image signals inputted from the image input terminal 1-2a are synchronized to clocks generated by the PLL circuit 8 and then converted to digital data by the AD converter 2. Digital image signals obtained by this process are stored into the memory unit B designated by 5-2 after being subjected to an image quality adjustment, image reducing conversion, and other processing in the input system image processing unit 2 designated by 3-2 and then transferred to the memory control unit 4.
The memory control unit 4 stores image data into the memory unit A designated by 5-1 at a timing corresponding to the input synchronizing signals (IHD1, IVD1) and the input system clock ICK1 on the basis of the signals made of processed IDATA1 and stores signals converted from IDATA2 to digital signals as image data into the memory unit B designated by 5-2 at a timing corresponding to the input synchronizing signals (IHD2, IVD2) and the input system clock ICK2. Furthermore, it reads out the both image data from the memory units 5-1 and 5-2 at a timing matching a size of a predetermined image synchronized with the output system clock OCK, the horizontal synchronizing signal OHD, and the vertical synchronizing signal OVD and a relationship of a display position and then transfers the data to the output system image unit 6. The image processing unit 6 adjusts the image quality or converts images with enlargement or the like. Accordingly, input images in various input system formats are converted to image signals in a format suitable for the image display unit 7 via the memory and the image data inputted from two inputs is synchronized onto a single screen for a multi-screen display.
Referring to FIG. 7, there is shown an example of operation timings at which image signals are input with (1) VGA (horizontal 640 pixels×vertical 480 pixels) 100 Hz and (2) SVGA (horizontal 800 pixels×vertical 600 pixels) 60 Hz as input signals (input signals in FIG. 6 or PC1 input or PC2 input in FIG. 13) when a resolution of the image display unit 7 is XGA (horizontal 1024 pixels×vertical 768 pixels) and the display vertical frequency is 75 Hz on the displays shown in FIGS. 6 and 13. For a description of FIG. 7, the operation of the PC1 input system is the same as for the PC2 input system relating to the device in FIG. 13 and therefore they are considered to be identical in description.
In FIG. 7, reference numerals 30, 31, and 32 designate a vertical synchronizing signal IVD (IVD, IVD1 or IVD2), a horizontal synchronizing signal IHD (IHD, IHD1 or IHD2), and a clock ICK (ICK, ICK1 or ICK2) of an input system for an input of VGA 100 Hz, respectively. One period of IVD is 1/100 sec and is equivalent to a period of (480+α1) pieces of IHD including α1 for a blanking period. One period of IHD is equivalent to a period of (640+β1) CLKs of ICK including β1 for a blanking period.
Reference numerals 33, 34, and 35 designate IVD (IVD, IVD1 or IVD2), IHD (IHD, IHD1 or IHD2), and ICK (ICK, ICK1 or ICK2) for an input of SVGA 60 Hz, respectively. One period of IVD is 1/60 sec and is equivalent to a period of (600+α2) pieces of IHD including α2 for a blanking period. One period of IHD is equivalent to a period of (800+β2) CLKs of ICK including β2 for a blanking period.
Reference numerals 36, 37, and 38 designate a vertical synchronizing signal OVD, a horizontal synchronizing signal OHD, and a clock OCK of the output system for an output of XGA 75 Hz, respectively. One period of OVD is 1/75 sec and is equivalent to a period of (768+α3) pieces of OHD including α3 for a blanking period. One period of OHD is equivalent to a period of (1024+β3) CLKs of OCK including β3 for a blanking period.
In this manner, the horizontal synchronizing signals, the vertical synchronizing signals, and input clocks of the input system have different periods according to resolutions. In the device shown in FIG. 6, the microcomputer unit 9 discriminates resolutions and formats from each other on the basis of the IHD, IVD, or the like and sets a dividing frequency ratio of the PLL circuit 8 to generate ICKs corresponding to each format. On the other hand, the output system operates with OHD and OVD generated at a certain counter value in the counter circuit 11 from the output clock OCK asynchronously with input system signals. In this manner, input system and output system vertical frequencies, in other words, screen update frequencies (frame rates) are converted.
In the device shown in FIG. 13, for the input system 1 (PC1 input), the input signal processing system is operated synchronously with IHD1, IVD1, and ICK1 and image data is recorded into the memory A. For the input system 2 (PC1 input), the microcomputer unit 9 first judges resolutions and formats on the basis of information transmitted with the IHD2, IVD2, or DDC2 and then makes ICK2 corresponding to each format to be generated by means of the PLL circuit 8. Next, the input signal processing system is operated synchronously with IHD2, IVD2, and ICK2 to record images into the memory B.
On the other hand, the output system is asynchronous with signals of the input system; OHD and OVD are generated at a certain counter value in the counter circuit from the output clock OCK and images of two systems are read out at a timing 2 synchronous with OCK, OHD, and OVD from the memory A and the memory B and synthesized before being transmitted to the output system image processing unit and the image display unit. As described above, a resolution is converted in addition to a conversion of a vertical frequency, in other words, a screen update frequency (frame rate) for the input system and the output system.
In this frame rate conversion, however, there is a problem of an image quality deterioration of dynamic images as shown in a specific example in FIG. 8. Also in a description of FIG. 8, the same phenomenon occurs relating to FIG. 13 because of the same operation between the PC1 input system and the PC2 input system. A description will be made here giving an example with a ratio of 5 to 4 as an input vertical frequency to an output vertical frequency (for example, 100 Hz to 80 Hz). In FIG. 8, there are shown five (a to e) continuous frame images 41 to be input and four (f to 1) continuous frame images 42 to be outputted to the display for the same period. Arrows indicate dynamic images moving from left to right on the screen.
Since an image for a screen is written into and read out from the same memory region, there is no change on the screen when a screen rewriting timing is accidentally close to a readout timing like (a) and (f) or (e) and (i), while a previous image is switched to the next one in a middle of reading out the screen like (g) or (h) causing a deviated view of the moving image in the upper or lower part of the screen when the screen is rewritten during the image is read out. This is called “frame tear”, which is a phenomenon of an image degradation in displaying a dynamic image. The same phenomenon occurs if the output vertical frequency is higher than the input vertical frequency to be contrary.
While this phenomenon is remarkable when a geometric object relatively large in comparison with the screen is translated horizontally, it is not so much conspicuous for a natural image and still images are frequently used in the conventional PCs with word processing, spread sheet processing, or plotting, and therefore the phenomenon has not been a serious problem. As described above, however, recently dynamic images are frequently displayed even on PCs and there has been an increase of screens on which geometrical graphics are moving, thereby causing higher requests for dynamic images.
As a countermeasure for the above problem, there is a double buffering method. In this method, not a memory region for a single image but memory regions for two images are prepared and these memory regions are alternately switched to each other for every screen in the write operation, and the memory region selecting operation is controlled so that scanning of the memory region under the readout operation is always ahead of scanning of the memory region for the write operation so as to avoid passing by an image in the readout operation.
For example, when data having the number of pixels of XGA (1024×768) is stored, input images of even-numbered fields such as fields m, m+2, m+4 and the like are stored into a first memory region at addresses 00000h to BFFFFh and input images of odd-numbered fields such as fields m+1, m+3, m+5 and the like are stored into a second memory region at addresses C0000h to 17FFFFh as shown in a memory map in FIG. 9. FIG. 10 shows a timing chart of the memory write and readout operations for the storage. Reference numerals 61 and 64 indicate an input vertical synchronizing signal (IVD) and an output vertical synchronizing signal (OVD), respectively. This description is also common to the PC1 input and the PC2 input relating to FIG. 13, and therefore it is assumed that IVD indicates IVD1 or IVD2 (or both) and that the first memory region and the second memory region are provided to the memory A or the memory B (or both), respectively.
Whenever the IVD 61 is entered, an input field is updated like m, m+1, m+2 and the like. Whenever the OVD 64 is entered, an output field is updated like n, n+1, n+2 and the like. Reference numerals 62, 63, 65, and 66 designate a signal indicating an execution of a write operation to the first memory region (WE1), a signal indicating an execution of a write operation to the second memory region (WE2), a signal indicating an execution of a readout operation from the first memory region (RE1), and a signal indicating an execution of a readout operation from the second memory region (RE2), respectively. They are shown here with active High.
In the write operation as described above, data is written into the first and second memory regions alternately between even-numbered input fields and odd-numbered input fields, but data is read out with selecting a field whose data is not displayed in the midst of the write operation. In this description, a vertical frequency of the output side is higher than that of the input side, and therefore the operation is controlled in such a way that data in the second memory region is read out if data of an input field is written into the first memory region when its output VD becomes High and that data in the first memory region is read out if data is written into the second memory region in order to prevent the passing by phenomenon. If the vertical frequency of the output side is relatively low, a control is required to prevent the passing-by phenomenon with considering a relationship between an input VD (IVD) and an output VD (OVD). In any case, as a timing for reading out data from the first memory and the second memory, memory regions are switched to each other in such a way that scanning of the memory region from which data is read out always precedes scanning for the write operation into the memory region according to input and output frequencies and a relationship between synchronizing signals.
In this double buffering method, however, the following problem on dynamic images as described below.
In the double buffering, the other memory region is selected so as not to display a memory region currently under the write operation. Therefore, for example, if there is a display showing continuous motions of a person who is rotating his arm as shown by input screens 71(a) to (d) in FIG. 11, it is possible to occur “frame duplication” in which the same image continues in two fields as shown in 72(e) and (f) among outputs whose frame rates have been converted like 72(e) to (i) or “frame omission” in which an equivalent field is missing like 82(c) among outputs whose frame rates have been converted like 82(e) to (g) if there is a display as shown by input screens 81(a) to (d) in FIG. 12.
As a method of improving a dynamic image quality different from the double buffering, there is a method of synchronizing the input vertical frequency to the output vertical frequency. For an XGA (1024×768 pixels) display unit, input signals of 50 Ha such as VGA (640×480) or SVGA (800×600) are converted to XGA of 50 Hz before displaying if the signals of 50 Hz are entered or input signals of 100 Hz are converted to XGA of 100 Hz before displaying if the signals of 100 Hz are entered in this method.
In these cases, a cycle time of the write operation and that of the readout operation to and from the memory matches each other, and therefore there is no problem on motions. If an input signal source has a low frequency such as 50 Hz, however, there is a problem on image qualities since the frequency is further reduced to a half in the same polarity for a device which displays a screen with an inverted polarity for each field like a liquid crystal display and therefore a screen rewrite cycle time is elongated, thereby causing a flickering phenomenon in which an entire view of the screen flickers. Furthermore, for a high frequency such as 100 Hz, a small number of pixels in VGA or the like, even if the speed is not so high in VGA, requires a speed of output approx. 2.6 times as high as a speed of an input when the number of the pixels in VGA is converted to one in XGA for the output, by which there is such a problem that an operation speed of the entire output system must be increased. For example, a clock rate of about 65 MHz for 60 Hz in XGA requires about 108 MHz for 100 Hz in XGA, thereby 100 MHz is exceeded. To solve this problem, a circuit configuration needs to be modified by adopting components having high operation speeds, developing new components, or dividing an operation system to reduce the speed, which leads to an increase of a cost or an expansion of a circuit scale.
In addition to the above problems, there are problems caused by two or more input systems. As shown by an example in FIG. 13, the input systems run synchronously with respective input signals, and therefore PC1 inputs are basically asynchronous with the PC2 inputs, while the output system outputs synthesized images from two systems at the same timing on a single screen, by which the input vertical frequency can be synchronized with the output vertical frequency for one system, but the synchronization cannot be applied to both systems simultaneously. This problem becomes serious for a use of multi-screen in which respective dynamic image qualities are important in further increased input systems.
Furthermore, for a plurality of image inputs, there occurs such a problem that a color, a brightness, a contrast or the like can be optimized so as to be suitable only for a single input system which is characteristic of a multi-screen in various image quality characteristics or that an optimization requires an increase of a circuit scale, in the same manner as for the image qualities of dynamic images.
Problems are described here relating to a gradation of an image quality such as a contrast, a brightness, or gamma characteristics of an image by giving an example of the circuit in FIG. 13.
Referring to FIG. 14, there is shown a gray scale signal for gradations horizontally changing in eight steps as one of video signals entered into the image processing device as shown in FIG. 13. There are shown a display screen in a gray scale in 14-1 and a signal for this display screen in 14-2. In 14-2, an abscissa axis corresponds to a time for a horizontal scanning period and an ordinate axis corresponds to a signal level. Reference numeral 14-3 indicates a horizontal synchronizing signal of an input signal for the display screen. In this example, the screen varies in eight gradations equally from 0% to 100% in a horizontal period.
In FIG. 15, there is shown a luminance characteristic of the display screen corresponding to the input signal in the image display unit 7 in FIG. 13. This characteristic is a transmission characteristic of light to an input voltage for a transmission LCD, for example, or a reflectance characteristic of light to an input voltage for a reflective device. Its appropriate characteristic depends upon the image display unit 7 in FIG. 13. It will be described here by giving two types of characteristics as in 15-1 and 15-2. In 15-1 and 15-2, the abscissa axis corresponds to an input signal level of the image display unit and the ordinate axis corresponds to a display luminance level with A, B, and C indicating input signal levels, respectively.
It is assumed here that input signals having different signal levels like 16-1A and 16-1B as shown in FIG. 16 have been inputted to the inputs IDATA1 and IDATA2 of the PC1 and PC2 systems in FIG. 13, respectively.
Furthermore, FIG. 16 shows display luminance levels 16-4 and 16-4B of signals in two systems for a use of an image display unit having the characteristic shown in 15-1 in FIG. 15. As shown by 16-4A and 16-4B, an input signal having the same number of gradations generates an image having a luminance level from 0% to 100% almost completely though there are collapsed parts of white 100% and black 0% in 16-4B of IDATA2 and on the other hand a whitish image due to a shift of black to a white side up to about 60% in 16-4A. Accordingly, when two images are displayed on an identical display simultaneously, there appears a mixed image made of two images having different black levels and different contrasts, by which the image display becomes very hard to see.
At this point, if each of the input system image processing unit 1(3-1) and the input system image processing unit 2(3-2) in FIG. 13 includes an auto-gain-control (AGC) circuit, a correction is made for signals having a DC level and an amplitude different from those of the input signals. If the image display unit is replaced with a display unit having the characteristic shown in 15-2 of FIG. 15, however, signals are not corrected for a change of the characteristic.
FIGS. 17 and 18 show input signals and luminance levels for characteristics of the display unit as shown by 15-1 and 15-2 in FIG. 15. In FIGS. 17 and 18, there are shown input signals 16-1A and 16-1B inputted to the inputs IDATA1 and IDATA2 of the two systems, signals levels 16-2A and 16-2B after passing the AGC circuit in the input system image processing unit 1(3-1) and the input system image processing unit 2(3-2), and luminance levels 16-4A and 16-4B of the display unit when these signals are entered. As shown by 16-4A and 16-4B in FIG. 17, the display has gradations from 0% to 100% due to a function of the AGC in the input-output characteristic of 15-1.
As shown by 16-4A and 16-4B in FIG. 18, however, the display has gradations only from 0% to 60% in the input-output characteristic of 15-2, thereby causing a whitish collapsed image in some gradations in the white side.
To cope with this replacement of the characteristics of the image display unit, the output system image processing unit 6 in FIG. 13 may have a correction characteristic for the characteristics of the image display unit. Referring to FIG. 19, there are shown input signals and luminance levels for the characteristic of the display unit of 15-2 in FIG. 15. There are shown input signals 16-1A and 16-1B inputted to the inputs IDATA1 and IDATA2 of the two systems, signals levels 16-2A and 16-2B after passing the AGC circuit in the input system image processing unit 1(3-1) and the input system image processing unit 2(3-2), signal levels 16-3A and 16-3B after passing the output system image processing unit, and luminance levels 16-4A and 16-4B of the display unit when these signals are entered.
By providing the correction characteristic for input signals to the in put system image processing unit for each signal and the correction characteristic for the display unit to the output system image processing unit in this manner, it becomes possible to obtain a multi-screen display which is not so much affected by a difference between input signal levels or by uneven characteristics of the display units. As correction characteristics, there are a brightness, a contrast, a gamma characteristic, and a white balance caused by a difference between color systems thereof.
Providing correction characteristics for image signals in both of the input system and the output system in this manner, however, leads to a need for preparing a plurality of similar circuits, thereby causing an enlargement of a circuit scale and an increase of the number of adjustment items which may cause an increase of a cost. It is a problem particularly in a configuration having a lot of inputs.
Furthermore, this configuration causes a degradation of an image since the image passes digital processing systems twice or more times for characteristics correction. FIG. 20 shows a conceptional diagram of assistance in explaining the degradation. In FIG. 20, the x axis indicates an input signal level and the y axis indicates an signal level of an output signal. This diagram shows an input-output characteristics of (4) y=[x2]*[x1/2]≅x in an 8-bit 256-phase digital processing system for (1) input-output characteristics of y=x after passing (2) a table having characteristic 1 of y=x2 and (3) a table having characteristic 2 of y=x1/2. While (4) should match (1) intrinsically, an operation error occurs whenever an output is standardized to 8 bits in each phase of passing operations for the characteristic 1 in (2) and the characteristic 2 in (3) and a bit error becomes large for y to x in a range of 0 to 50 or so in (4) after the synchronization.
If the above is applied to a conventional example, it is equivalent to giving a characteristic conversion like (2) in the input system and giving a characteristic conversion like (3) in the output system. In this case, gray-scale level of the black level of an output image is deteriorated, thereby causing a degradation of an image quality such as a pseudo-outline in the image.
Although there is a method of increasing an operational bit count to prevent this phenomenon, it causes an increase of a cost or an enlargement of a processing system.
As described above, there has been conventionally a problem of an increase of a cost due to an enlargement of a scale in a configuration in which characteristics of different input images are converted to the same display characteristics for a change of characteristics of a display unit in a multi-screen display in which images from a plurality of input signal sources are displayed in the same display unit. In addition, a circuit has a serious bit error in the circuit configuration, which leads to a problem of a degradation of an image quality. Therefore, a brightness, a contrast, gray-scale level, colors or the like cannot be easily unified among respective input systems.
If an AGC circuit is included in the input image processing device, a dynamic range of signals can be assured, but signals are automatically corrected and therefore signal levels which should be displayed originally are also corrected, which causes a problem that an intention of a signal transmission side is ignored.